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HIGH-SPEED CCD IMAGING, 
IMAGE PROCESSING AND CAMERA SYSTEMS 



5 CROSS REFERENCE TO RELATED APPLICATION 

This application claims the benefit of and 
priority from pending U.S. provisional application no. 
60/017,035, filed May 3, 1996. 

10 

GOVERNMENT RIGHTS 

The invention was made with Government support 
under Contract No. F08630-96-C-0066 awarded by the 
15 Department of Defense. The Government has certain 
rights in the invention. 



BACKGROUND OF THE INVENTION 

20 

The present invention relates generally to 
very high speed imagers, optical processors and 
electronic camera systems. More particularly, the 
present invention relates to electronic imaging systems 

25 which are capable of electronic image acquisition at 
image frame rates for example, up to one million image 
frames per second or more, as well as camera systems 
having extended compact analog storage capacity and/or 
processing capacity for acquired images. 

30 There is a need for economical, ultrafast 

electronic imaging in a variety of scientific, 
commercial and military applications. Ultrafast 
electronic imaging sensors with up to million-f rame-per 
second or more recording speed would be desirable for 

35 fundamentally enabling applications in particle physics, 
high speed chemical physics, industrial process control, 
biomedical applications, high resolution long range 
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imaging telescopes/ and combustion flow fields. 
Unfortunately, current electronic detectors are not 
adequately sensitive, are relatively slow, or are 
expensive. For example, currently available high 
5 resolution electronic imagers, such as commercial CCD 
image sensors fabricated in silicon, have practical 
speed limitations on the order of several thousand 
frames per second for moderate to large frame sizes 
(e.g., at least 512 x 512 pixels) using a moderate 

10 number of output channels (e.g., up to 32). Some 
electronic imager frame rate speed improvement is 
possible with faster semiconductor devices, such as 
bumped GaAs output circuitry, but large frame rate 
increases will require new imager designs and 

15 geometries. 

Conventionally, providing higher effective frame 
rates generally involves multiple imagers in multiplexed 
assemblies, or high speed film systems. In this regard, 
various ultrahigh speed cameras are based upon rotating 

20 prism/mirror assemblies which have an inherent 

mechanical time delay for start up and synchronization. 
These systems are large, inflexible and expensive, and 
have a variety of disadvantages. 

For example, achieving high rotational speed of the 

25 mirror/prism assembly may require that the camera 

housing be purged with helium to reduce drag, and that 
the prism be made of relatively brittle beryllium, which 
may present a health hazard. Because of the mechanical 
inertia involved, such camera systems typically may be 

30 limited to triggering the event to be imaged (rather 

than being triggered), and typically provide only fixed 
interframe periods. The mechanical ''spin up" time 
required for the camera to reach steady state makes it 
difficult to asynchronously trigger the camera based 

35 upon an event to be imaged. This also complicates 



WO^/46004 




PCT/US97/07762 



-3- 

simultaneous triggering of multiple cameras for 
stereoscopic image capture, A user selectable 
interframe period combined with asynchronous triggering 
and fast gating control would permit capture of image 
5 data at high temporal resolution, and would also provide 
for stereo photogrammetry through the use of common, 
precision triggering of multiple cameras. It would be 
desirable to provide control of electronic image 
exposure separately from image capture frame rate, to 

10 permit more flexible matching of camera performance to 
image capture needs of a particular imaging application. 
However, at ultrahigh imaging speeds, independent 
control of shuttering and frame rate is difficult to 
provide together with high image quality. 

15 Managing and storing the image output data 

from an ultrafast imager is also a current problem. For 
example, the image data capture and memory requirements 
for a IK X IK pixel camera operating at one million 
frames per second are enormous. The digital output 

20 image data rate of a IK x IK pixel imager operating at 1 
million frames per. second is one terabyte per second, 
which exceeds the capability of presently available 
digital storage systems. 

25 SUMMARY OF THE IM\^EI:TI0N 

A primary object of the present invention is 
to provide high-speed electronic ima-jers which are 
capable of high frame rates, such image acquisition 

30 frame rates of up to one million frames per second or 
more. It is a further object to provide imagers and 
camera systems having novel and effective geometries, 
which facilitate efficient image acquisition. In 
addition, it is also an object to provide independent 

35 selection of frame rate and exposure time at such very 
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high frame rates. Yet another object of the present 
invention is to provide compact image storage capability 
for ultrahigh speed image acquisition. 

Generally, the present invention is directed 
5 to electronic image sensors and electronic cameras 
systems which are capable of extremely rapid image 
acquisition, which also comprise high speed analog 
memory which is locally linearly associated with the 
imaging pixels of the high speed imager to store the 
10 images for a sequential plurality of acquired images. 

BRIEF DESCRIPTION OF THE DRAWINGS 

These and other objects of the present 
15 invention will be apparent from the following detailed 
specification and the accompanying drawings, in which: 
Figure 1 is a schematic block diagram of an 
embodiment of a CCD imaging system which utilizes an 
ultrahigh frame speed, low fill-factor CCD imager having 
20 a plurality of analog memory sites directly linearly 
associated with each photo-detecting site; 

Figure 2 is a schematic plan view of one 
embodiment of an ultrahigh frame speed CCD imager which 
may be used in an imaging system such as that of 
25 Figure 1; 

Figure 3A is a schematic top view of a portion 
of the imager of Figure 2 showing the spacing of active 
photosites with respect to the locally linearly 
associated memory storage cells; 

30 Figure 3B is a schematic plan view of an 

alternative photosite and local memory geometry for the 
imager of Figure 2; 

Figure 4 is an enlarged plan view of one of 
the photosite sections of the imaging array of Figure 2; 

35 Figure 5 is a cross-sectional view of the 



W0 97A 



wo 97/46004 




PCT/US97y07762 



-5- 

active photosite of Figure 4 taken along the horizontal, 
row direction of Figure 2; 

Figure 6 is a cross-sectional view of the 
active photosite of Figure 4 taken along the vertical, 
5 transfer direction; 

Figure 7 is a schematic top view of a 3-D 
stackable CCD sensor for an ultrafast imager with 
greatly extended analog memory directly associated with 
each imaging photosite; 
10 Figure 8 is a 3-D stacked array of 

edge-imaging CCD sensors of Figure 7, with IK buffer CCD 
channels, forming a compact 2D imager and 3-D analog 
memory system; 

Figure 9 is a bump-mounted high speed CCD 
15 imaging system, with extended 3-D memory and/or 
processing capacity; 

Figure 10 is a cross-sectional side view of a 
multiplexed SOS imager bump-mounted to a CCD wafer stack 
of the type shown in Figure 9; 
20 Figure 11 is a schematic cross sectional side 

view of "pillar-recess" z- and y-axis alignment system 
for a 3-D image sensor and memory assembly such as that 
of Figure 8 or Figure 9; 

Figure 12 is an ultrahigh speed stereo 
25 photogrammetry system using a plurality of 

simultaneously triggered ultrahigh speed cameras of the 
present invention; 

Figure 13 is a schematic top view of a large 
format 4096 x 4096 storage cell/pixel CCD imager of full 
30 frame architecture which is adapted for ultrahigh speed 
imaging; 

Figure 14 is a top view of an ultrahigh speed 
CCD imager with a four phase, high conductivity, 
parallel transport electrode bus system; and 
35 Figure 15 is a schematic plan view of a 
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portion of an ultrahigh speed imager like that of Figure 
3A, which is adapted for color imaging. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

5 

Generally, the present invention is directed 
to electronic imaging systems comprising an array of. 
imaging photosites, having locally associated therewith 
a storage plurality of linearly sequentially operable 

10 analog memory cells for storing sequentially captured 
image charge signals from the respective photosite. 
Generally, the photosite array will desirably be 
two-dimensional array of at least 1000 active photosites 
regularly spaced in a two-dimensional array, and 

15 preferably may be configured as a regular two- 
dimensional array of at least 200,000 active photosites- 

As indicated, the photosites each have a 
storage plurality of locally associated, sequentially 
operable analog memory cells. By ''locally associated" 

20 is meant each photosite in the regularly spaced imaging 
array is directly connected electrically, such as by 
means of a CCD charge transport channel, to a storage 
plurality of at least 4, preferably at least 10, and 
desirably at least 64 linearly adjacent, electrically 

25 connected, sequentially transf errable, analog memory 
cells for each photosite. In addition, analog memory 
cells which are linearly locally associated with 
adjacent photosites are preferably adjacently positioned 
along a charge transport channel, such that the groups 

30 of analog memory cells which are locally associated with 
linearly adjacent photosites may also be transferred and 
read out sequentially. Desirably for highest speed 
operation, each photosite will be directly connected to 
at least 16 sequentially operable analog memory cells 

35 which can receive and store sequential charge signals 
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samples generated by the photosite at a rate of at least 
about 100,000 image samples per second and preferably at 
a rate of at least 200,000 samples per second, and most 
preferably, at least 1,000,000 image samples per second. 
5 In one preferred embodiment, the group of 

analog image signal storage cells locally associated 
with each photosite are positioned linearly and 
sequentially along continuous CCD charge transport 
channels. The respective groups of analog image signal 
10 storage cells of linearly adjacent photosites are also 
positioned regularly and sequentially along each 
transport channel, so that the groups of image signals 
locally linearly associated with adjacent photosites may 
be read out of the transport channels in the same 

15 relative order. Desirably, all of the analog image 
storage cells may be shifted in parallel by a common 
transport electrode array, which will preferably be a 
three or four phase electrode transport clocking system. 

Solid state CCD imagers conventionally utilize 

20 a regularly arranged array of photodetecting sites which 
produce and collect electrical charge when exposed to 
light. The amount of charge collected as a charge 
packet at the photodetection site corresponds to the 
intensity of the image and the integration time at the 

25 respective photosite location. These charge packets are 
then periodically transferred from the imager for 
digitization and further processing. The electronic 
imager photosites may be those utilized in a variety of 
photosensor systems such as a surface channel or buried 

30 channel charge coupled devices ("CCD") or other charge 
transfer device ("CTD"), a charge injection device 
("CID"), or other photosensor array with which analog 
memory may be locally associated, as will be described 
in more detail. Although imaging photosites designed 

35 for the visible spectrum (e.g., 400-700 nanometers in 
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wavelength) , such as silicon photodiode imagers are 
particularly desirable, the present methods and 
apparatus are also useful for imaging in the infrared 
(e.g./ 0.7 to 15 microns), far infrared (e.g., 15-25 
5 microns, or 15-100 micron wavelength), and ultraviolet 
(e.g., 100-400 nanometer wavelength) spectral ranges 
using silicon and other semiconductors such as III-V and 
II-VI materials. Various types of imagers are 
conventionally known and used over various such light 

10 wavelength ranges, and such imagers may be used to 

provide active photosite detectors at such wavelengths 
in accordance with the present invention. Charge 
coupled devices having buried channel charge transfer 
channels are particularly preferred because of their 

15 high speed operation and compact, sequential memory 

storage capability when employed with sensor geometries 
of the present disclosure. Such CCDs and associated 
camera drive and output electronics may be provided . 
which are capable of very high frame rates, good pixel 

20 uniformity, and low noise when used with the imager 
geometries and operating methods of the present 
disclosure. 

High speed Charge Injection Devices (CIDs) and 
CMOS photosites may also be used as electronic imagers, 

25 particularly for edge-sensitive photosites for 3-D 
stacks of imaging-memory chips as will be described. 
Such devices may comprise a linear array of 
photosensitive capacitor elements at the edge of the 
chip. Image readout may be performed by storing pixel 

30 image charge on one electrode and subsequently measuring 
the potential produced at the other electrode upon 
transfer of a charge packet to that electrode [see 
Zarnowski et al., SPIE Vol. 1447 (1991) Charge Coupled 
Devices and Solid State Optical Sensors II, pp. 191-201; 

35 Carbone et al., same SPIE Vol. 1447, pp. 229-237 for CID 
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imager description] for transfer into an extended CCD 
memory associated therewith. CTDs and CIDs may also be 
designed to perform image subtraction at the local pixel 
level on a charge mode basis at each pixel and other 
5 local neighborhood operations to provide the sampled 
image input. The photosite array may desirably have an 
effective fill factor of at least 75 percent when 
provided with a lenticular lens array or a lenslet array 
(e.g., diffractive optic or graded index). 

10 It is desirable that design of the optical 

lens system and the electronic imager system be 
optimized to provide an appropriate balancing of 
resolution and aliasing. [C. S. Bell, Lens Evaluation 
for Electronic Photography, SPIE Vol. 1448 (1991) 

15 pp. 59-67). In this regard, the limiting resolution of 
the image sensor and its inherent aliasing character- 
istics should best be designed to correspond with the 
characteristics of the lens system. A large amount of 
image information is lost in conventional imaging 

20 systems as a result of aliasing and/or blurring, even 
under optimal conditions. A minimum of two pixels 
is needed to detect modulated contrast in an image 
{although it is noted that each individual lattice pixel 
can produce an image edge enhancement value of the image 

25 at the image lattice position in accordance with the 
present invention) , so the Nyquist frequency limit of 
the sensor array is approximately inversely proportional 
to twice the pixel repeat distance. Aliasing, an 
interference effect, may occur in imagers with image 

30 information at frequencies (spatial resolutions) higher 
than the Nyquist limit. Optical blurring is conven- 
tionally used with single frame imagers to reduce (or 
match) the lens spatial frequency resolution to that of 
the imaging system. For systems with color sampling, 

35 optical prefilters providing different amounts of blur 
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or smear for different colors may be utilized to 
optimize the lens system to the image at each wavelength 
sensitivity in accordance with conventional practice. 

To provide increased resolution, imagers are 
5 conventionally used in an image scanning mode in which 
successive images are made after moving the imager by a 
pixel width in the image field {"step and repeat" or 
"fine scan" operation) . Imagers used to increase image 
resolution in this manner are also intentionally 

10 designed to have low fill factors [J. Milch, "High 

Resolution Digitization of Photographic Images With An 
Area Charge-Coupled Device (CCD) Imager", Applications 
of Digital Image Processing, Proc. SPIE 697, pp. 96-104 
(1986) ; Kontron Commercial Camera] . Accordingly, it 

IS will be appreciated that the methods and apparatus of 
the present disclosure may use systems in which the 
output frames of the photosensor sites of an ultrafast 
imager are shifted at high speed to form a higher 
resolution, composite image. 

20 Illustrated in Figure 1 is an ultrahigh speed 

electronic camera system 100 which utilizes the overall 
high-density design of a relatively large conventional 
frame transfer or full frame CCD imager which has been 
modified in its photosite architecture by means of a 

25 specialized photomask, to permit high-speed image 

capture operation. Large pixel count CCD sensors such 
as that employed in the system of Figure 1 are typically 
designed for relatively low frame rate applications, 
such as from about 1 to about 60 image frames per 

30 second. While all of the columns of the image data may 
be conventionally shifted in parallel at high speed 
(e.g., over a million shifts per second along the 
vertical transfer registers), the conversion of the 
parallel data stream to serial output through a small 

35 number of output circuits (e.g., typically from 1 to 4 
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output channels, although 32 or more channels may be 
used for specialized systems) limits the frame imaging 
speed of such imagers, CCD imagers of up to 5K x 5K or 
more pixels (e.g., 25-60 million pixels) are 
5 conventionally available in a full frame architecture. 

As schematically illustrated in Figure 1, the 
ultrahigh speed imaging system 100 comprises a lens 102, 
an electrooptical shutter and lenslet system 104, a high 
speed CCD image sensor 106, an electronic drive and a 

10 control system 108 for the CCD image. sensor 106 and 
electrooptical system 104. The lens 102 may be any 
suitable camera lens for focusing a coherent or 
incoherent image or optical transform effectively at the 
focal plane of the imager 106. The optional 

15 electrooptical system 104 may include a high speed 
electrooptic shutter such as a Kerr cell, and/or a 
lenslet array to increase the effective fill factor of 
the high speed CCD image sensor 106, which will be 
described in more detail with reference to Figures 2-11. 

20 The electronic drive and control system 108 may also be 
of substantially conventional design, for supplying the 
drive and output signal processing for the imager 106, 
and yet provide image frame rates on the order of one 
million frames per second for a continuous sequence of 

25 frames, when used in conjunction with the imager 

modifications which will be described in connection with 
Figures 2-6. The CCD drive and control system 108 
generally comprises vertical (parallel) shift register 
drivers, horizontal shift register drivers, data output 

30 A/D conversion and memory circuitry, CCD biasing 

circuitry and other circuitry appropriate to controlling 
a particular CCD imager (see ''CCD PRODUCTS" 1996, the 
1996 product sales catalog of Thomson-CSF 
Semiconducteurs Specifiques, for examples of CCD imagers 

35 and connection pins, register drive and readout voltage 
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10 



15 



20 



25 



30 



and timing diagrams, and CCD camera/driver kits, which 
is incorporated by reference herein) 



frame transfer 1024 x 1024 CCD imager 106, which 
features a parallel architecture which is easy to drive 
and to collect the output data. The imager 106 has 1024 
X 1024 closely adjacently packed photosensitive pixels 
of 14 microns x 14 microns cell dimension, repeated in a 
square array. The 1024 x 1024 pixel . imaging zone is 
positioned along conventional vertical shift. registers 
next to a light-shielded frame transfer memory zone. In 
accordance with this conventional design, the entire 
1024 x 1024 pixel image frame may be rapidly transferred 
to the shielded frame transfer zone at a parallel frame 
transfer rate of about 1.3 Mhz, or less than 0.001 
second to transfer the 1024 lines of the image from the 
image zone along the vertical transfer columns of the 
device. The shielded frame transfer memory zone has 
four parallel outputs running at 20 Mhz maximum data 
rate each, which provides full chip output rates as high 
as 60 Hz (made of 1024 x 1024 memory cells) with 10-12 
bit gray scale resolution. Other specific on-chip 
capabilities include anti-blooming and exposure time 
control. The illustrated device 106 is a commercial THX 
7887A CCD imager of Thomson-CSF Semiconducteurs which is 
conventionally operated at up to 60 frames per second, 
but which is modified by a specialized light shield, and 
the method of operation, to provide a structure for 
ultrafast imaging at a frame rate in excess of one 
million frames per second. 



square frame transfer area array CCD imager which is 
specially designed to capture up to 16 images of 256 x 
255 pixel image frame size, at frame rates up to 
approximately 1.3 megaframes per second. The special 



Illustrated in Figure 2 is the large format 



As shown in Figure 2, the CCD imager 106 is a 
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modification comprises a metal mask over the imaging 
section which is opaque except for a precisely aligned 
pattern of pixel openings which permits the ultra 
high-speed operation of the imager. The imager 106 
5 itself comprises an approximately 1 megapixel imaging 
zone 202 formed by 1024 x 1024 square pixels (14 microns 
X 14 microns) arranged along vertical CCD shift 
registers (in the direction of the arrows) and a memory 
zone 204 having the same storage capacity, allowing for 
10 simultaneous and controlled readout of the memory zone 
analog data. The image zone 202 comprises 1044 
horizontal lines (rows), including the 1024 pixel lines 
and five dark reference lines at the top and five dark 
reference lines at the bottom of the array 202. 
15 Supplementary lines are used as isolation lines for dark 
reference integrity. The optically inactive memory area 
204 has the same number of lines as the imaging zone. 

In order to increase the output rate of the 
memory zone 204, the imager has four output sections 
20 206, 208, 210, 212, each of which receives the analog 
charge packets from one fourth (256) of the 1024 CCD 
transfer columns. Accordingly, the imaging and fast 
frame storage zone 202 and the memory zone 204 may be 
regarded as being functionally divided into four 
25 sections of 256 CCD transport channels each. The 
imaging and fast frame storage zone 202 itself is 
specially designed to permit ultrafast acquisition of 
sixteen frames of 256 x 255 pixel image size, by 
specialized masking of the otherwise actual imaging 
30 zones of the imager, together with high speed transport 
clocking of the CCD transport channels of the imager. 

A plan view of a portion 300 of the imaging 
and fast frame local analog memory storage zone 202 of 
the CCD imager 106 is shown schematically in enlarged 
35 form in Figure 3A. As shown in Figure 3A, the otherwise 
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active pixel cells 302, 304 are arranged in conventional 
vertical CCD transport channel columns oriented in the 
direction of the arrows, toward the memory zone 204 (not 
shown) of the frame transfer imager 106. Each of the 
5 cells 302, 304 would ordinarily be active photosites 
for imaging through the respective four-phase vertical 
transport register electrodes of the imager 106 in 
accordance with conventional practice (see Figures 4-6) . 
However, the otherwise conventional CCD imager 106 is 

10 modified by a specialized opaque metal mask 306 shown as 
a dark trellis pattern, which covers the majority of the 
photosites 304. 

In the illustrated embodiment 300," only one 
sixteenth of the cells are not masked by the opaque 

15 aluminum layer. The pixel openings in the mask layer 
306 which form the active photosites 302, are arranged 
regularly over the surface of the imager (spaced 
approximately 4 cells apart vertically and horizontally) 
but angularly displaced with respect to the longitudinal 

20 direction of vertical transfer registers 310, such that 
charge collected at a photosite 302 may be linearly and 
serially shifted 15 times vertically along the transport 
channels 310 through the "opaque" storage cells 304, 
before reaching another active photosite 302. 

25 As shown in Figures 3A and 3B, the active 

photosites 302 are regularly arranged with respective 
predetermined spacings along two axes, at least one of 
which is rotationally displaced with respect to the 
longitudinal direction of the vertical (parallel) 

30 transfer registers 310 of the CCD 106. This permits the 
linear shifting of image data from each photosite to 
progress a multiple of the predetermined spacing between 
the active photosites 302 generally along the transport 
channel direction, before encountering another active 

35 photosite 302. The predetermined spacings will 
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generally correspond to a whole number of pixel/memory 
cells 302, 304 in the direction of the respective axis, 
and may be different for each axis of the photosite 
array. 

5 As shovm in Figures 3A and 3B, the multiple of 

the spacing along the transfer channel direction is a 
function of the integer cell spacing along the other 
axis of the photosite array. Accordingly, the photosite 
and storage cell arrangement shown in Figure 3A is 

10 capable of capturing and storing 16 images. If the 

photosites were spaced apart by 8 cell units along each 
array axis, the imager would be capable of capturing and 
storing 64 images, but the number of active pixels in . 
the imager would be one fourth of that with 4 cell 

15 spacing. 

Because the charge packets may be transported 
very rapidly along the transport channels (e.g., at four 
phase clocking speeds of over 1 Mhz in accordance with 
conventional frame transfer operation of the TPDC 7887A 

20 CCD imager and camera drive circuitry) , the imager 

design 300 can accomplish extremely rapid imaging while 
transporting the image charges. The 16 successive 
images are interleaved as shown in Figure 3, with 
cells 16 being the first image, cells 15 being the 

25 second image, back to cells 1 beincr the most recent 

image. The 16 images captured and stored in the imaging 
and fast framing storage zone 202 nay be clocked to the 
memory zone 204 for subsequent readout, using 
conventional electronic camera CCD drive and output 

30 circuitry for the Thomson 7887A CCD iriager chip. 

In operation, the ultrahigh speed camera 
system 100, under control of the electronic drive and 
control system 108, may initially have shutter 104 in 
closed position/ and the four phase transport electrodes 

35 will be in quiescent, nontransfer mode which is 
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receptive to image capture. At least two, and 
preferably three of the four electrodes are placed at 
electron depletion potential, to maximize the image 
signal charge storage capacity of the photosites. 
5 Preferably^ prior to image capture, the antiblooming 
gate and drain are activated to remove unwanted image 
signal • 

To capture an image at high speed in a 
relatively simple mode of operation, the system 108 

10 opens the shutter 104 and/or activates the illumination 
of the event to be imaged, and drives the four phase 
transfer electrodes of the vertical registers up to 16 
cycles of parallel, vertical transfer along the vertical 
registers 310 of the imaging zone 202, at a vertical 

15 shift rate corresponding to the desired image capture 
rate, which may be at a rate of at least 50,000 frames 
(or vertical shifts) per second, up to one million or 
more frames per second • The initiation of the transfer 
cycling may be precisely triggered within less than one 

20 quarter of the cycle time, in accordance with 

conventional imager drive practice. The antiblooming 
and drain gates are deactivated during imaging. 

The THX 7887A imager 106 is designed to 
operate at vertical shift rates of up to about 1.3 

25 million cycles (vertical shifts) per second. The 

maximum number of images which may be captured at high 
speed is limited to the number of memory cells which are 
locally associated with each active photosite, here 16. 
The vertical clock shift cycles are then 

30 stopped, while maintaining the image data in the 
respective storage cells. The shutter or scene 
illumination may be then be turned off, and the 
composite images clocked to the frame store zone 204, 
from which they may be read out at high precision in the 

35 conventional manner. The individual images, having a 
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precisely known location in the composite image, may be 
easily ^'sorted out" or separated in digital memory in 
any appropriate manner. While the imaging may be 
carried out as previously described to produce uniform 
5 frame rates in which the exposure time is the reciprocal 
of the frame rate, other modes of image capture may be 
used. 

For example, the exposure time of each image 
may be separately programmed and cycled, if desired. In 

10 this regard, with reference to Figure 3A, the first 

transfer cycle time could be one microsecond, the second 
through fifth cycle times could be 5 microseconds, and 
the remaining sixth through sixteenth cycle times could 
be 10 microseconds, with corresponding exposure times. 

15 Similarly, the camera may be operated to 

provide independent control of frame rate and exposure 
time, such that the image exposure times may be 
effectively less than the reciprocal of frame rate. For 
example, by clocking the CCD vertical transfer to shift 

20 the first image at a first rate (e.g., one microsecond) 
then hold the second image for a predetermined exposure 
time (e.g., 9 microseconds) before shifting the image 
again in one microsecond, followed by an image shift for 
the third image at the first (one microsecond) rate, and 

25 the fourth image at the 9 microsecond exposure time, 
etc. By alternating images at a first exposure time 
with images at a second exposure time, alternate images 
may be discarded to provide the desired exposure 
control. In the example given, the effect of 100,000 

30 frame per second imaging with either one microsecond or 
9 microsecond exposures is provided in a completely 
programmable and user selectable manner. The penalty is 
that approximately half of the images which are 
discarded, and the corresponding storage area of the 

35 imager is not directly utilized. 
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However, this is a very powerful feature of 
the present disclosure^ which is programmable via 
control of imager clocking, and does not require custom 
imager chips, such as very large interline transfer CCDs 
5 with independently gated photosites. 

The active photosite arrangement of Figure 3A 
has rows of photosites arranged to permit extended 
linear shifting of charge packets in linear memory cells 
locally associated with the active photosites, but this 

10 arrangement is not a square pixel array. Illustrated in 
Figure 3B is an active photosite distribution which does 
present a square active pixel array, albeit one in which 
both axes of the array are rotated with respect to the 
vertical charge transport channels 310. It should be 

15 noted that in this arrangement, in which successive 
active photosites 302 are spaced approximately four 
cells apart but rotationally displaced both horizontally 
and vertically with respect to a rectangular grid 
aligned with the CCD transport channels (in the 

20 arrangement of Figure 3A the active photosites are only 
rotationally displaced in one direction), that the 
imager can store 17 rapidly successive images, rather 
than 16 images. 

Similar patterns may be provided to further 

25 increase the number of successive images which may be 
successively captured and stored in analog memory cells 
locally adjacent the active photosites. For example, an 
array of regularly spaced photosites line that of Figure 
3A, but in which the active sites are separated by eight 

30 cells, will provide an array in which an analog charge 
may be shifted 64 times before it encounters another 
active photosite, and accordingly the system can capture 
64 successive images at extremely fast image frame 
rates. 

35 A similar square array like that of Figure 3B, 
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in which the active photosites are displaced in both the 
horizontal and vertical directions and spaced apart 
approximately eight cell units, will increase the number 
of successive image frames to 65. This also results in 
5 approximately one fourth of the resolution in each image 
frame, given the same number of cell units, as 
previously discussed. For this reason, it may be 
desirable to use even larger imager sizes, such as 2024 
X 2024 or 4048 x 4048 cell imagers, in order to provide 
10 for extended image frame numbers, while maintaining a 
desired degree of image resolution. 

The imager 106 is used to output the 16 
(Figure 3A) or 17 (Figure 3B) image frames stored in the 
frame memory output area 204 at a rate of up to sixty 
15 frames per second (fps), in the conventional manner for 
this chip. This frame rate is obtained through four 
parallel area arrays of 256 (V) x 1204 (H) pixels each, 
each ended by its own serial CCD readout register, 206, 
208, 210, 212. Four parallel output amplifiers deliver 
20 the data signals to the control unit 208 at the maximum 
data rate of 20 Mhz (4 X 20 Mhz for the whole array) . 
The output data is digitized and stored in digital 
memory in the control unit 108, where the individual 
images are separated. 
25 For 1 million frame per second operation the 

electronic image of the photon flux impinging on the 
unmasked photosensitive areas is sampled by the pixel 
array during a time period which is very short. 
Electronic shuttering is available on the chip, but is 
30 not necessary during this period. However, if it is 
desired to operate the chip at slower frame rate, this 
can be done by clocking the anti-blooming gate to the 
shuttering level voltage, then dumping the unwanted 
photogenerated charges towards the anti-blooming drain. 
35 After the 16 frames are captured, at a continuously 
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uniform rate, or at a programmed varying rate, the 
comjxjsite electronic image may be quickly shifted from 
the imaging area towards the memory area at very high 
speed. During this period, the electrooptic shutter 104 
5 may be turned off (rendered opaque) , the illumination 
may be turned off, or on-chip electronic shuttering may 
be used when available. In this way, the charge packets 
each representing an exposed image pixel of one of the 
16 images taken in rapid succession, are all transferred 

10 in the same order to the memory zone 204. 

The memory zone 204 can be shifted 
independently of the imager zone, a line by a line, into 
the four serial CCD registers, each of which is termi- 
nated by an output node (floating diode) . These output 

15 nodes are simultaneously reset to the reference voltage 
level VDR prior to charge packet receipt from the next 
CCD serial shift register stage. Their potentials are 
permanently sensed by a double stage source follower 
nMOS amplifier, and sampled charge packets in the CCD 

20 register are then clocked continuously, stage by stage, 
towards these nodes. Each output amplifier accordingly 
delivers the electronic charge information of a quarter 
of an image line-256 pixels-, in the serial typical 
waveform of a CCD output floating diode signal, for the 

25 16 interleaved ultrafast images stored in the frame. 

As shown in Figure 4, the unit cell pitch of 
the photosite and storage cells 302, 304 is 14 microns x 
14 microns, and includes the anti-blooming structure and 
sensitive area which serves as a photosite at the mask 

30 openings. In this frame transfer architecture, the 

sensitive area 302 acts also as the locally associated 
analog memory for the active photosite, as the inactive 
storage cell units 304 for those cells which are masked, 
and as the transport medium towards the memory zone--see 

35 Figure 2. The design of this area should optimize the 
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"fill factor" (FF) of the pixel and its light respon- 
sivity/ its saturation charge^ the efficiency of the 
ant i -blooming drain, acting also as a dump drain for 
integration time control (electronic shutter) and the 
5 transfer efficiency at high speed. The fill factor of 
the individual active photosite is still relatively 
small compared to the area of the imaging zone 202 
because* of the masking utilized to permit high speed 
operation. The effective active photosensor area may be 

10 increased by lenticular microlenses and/or refractive 
lenslet arrays, such as may be included in the optical 
component 104 of Figure 1. By providing, for example, a 
refractive microlens array of individual lenses of area 
approximately 10 to 15 times the area of the active 

15. photosites 302, which are respectively each perpen- 
dicularly above and focused on each of the active 
photosites, the effective fill factor may be increased 
to at least 80 percent. Increasing the effective fill 
factor, and the corresponding effective light efficiency 

20 with respect to the image is particularly important for 
ultra high imaging speeds, which tend to require high 
light intensities to permit collecting enough light to 
register a useful image in any event. 

The cross-view of the pixel is shown in Figure 

25 5. As shown in Figure 5, the pixels have a lateral 

overflow drain structure which may be used in the high 
speed operation of the imager, as discussed previously. 
The anti^blooming structure is shared between two 
neighboring pixels, acting as the pixel separation on 

30 the both horizontal pixel sides. This structure is made 
of half a drain and one gate on each side of the pixel 
(anti-blooming drain and anti-blooming gate), that is 
one drain and two gates per pixel pitch. The potential 
barrier controlled by the anti-blooming gate has to be 

35 set to the anti-blooming potential level so that 
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exceeding charges in the CCD channel spill out first 
into the anti-blooming drain instead of blooming up and 
down in the neighboring pixels. 

The photosensitive and transport area is 
5 located in the CCD part of the cell units. It is made 
of a set of four overlapping gates as shovm in Figure 6. 
This allows a four phase transfer operation during the 
image shift towards the memory zone. This transfer mode 
is used for its good charge handling and charge 

10 capacity/ largely greater than the three phase or two 
phase mode. As an example, the saturation level of this 
four phase pixel-cell unit is 210,000e-, when it would 
be only 140,000e- for the three phase mode. Moreover, 
the four-phase structure allows charge collection and 

15 integration in a potential well controlled by three 

gates-about 10.5 microns long in the 14 microns pitch-, 
longer than the two-gate well control-9.3 microns long 
for this pixel-when using the three gate architecture, 
which improves the optical modulation transfer function 

20 of the sensor. 

Each 14 X 14 micron square imaging pixel uses 
a three level polysilicon MOS gate structure on a buried 
channel phosphorus implantation in an epitaxial p/p+ 
silicon substrate. The anti-blooming overflow drain is 

25 phosphorus implanted and self-aligned on the two 

controlling gates. Owing to the 1.5 microns design rule 
CCD technology, this anti-blooming structure layout is 
only about 5.5 microns wide. The pixel aperture, at the 
Si/Si02 interface level, is estimated as 10.3/14, 

30 providing a fill factor of 0.74 for the active cell. 

This value is the minimum fill factor of this pixel for 
the blue wavelength, as blue photons are absorbed in the 
first hundred nanometers under the interface. For red 
light, the fill factor is larger, as the internal pixel 

35 separation inside the silicon is thinner than it is at 
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the Si/Si02 interface/ the internal electrical fields 
expanding under the anti-blooming gate, to increase the 
fill factor for these wavelengths. 

In the memory zone, vertical isolation between 
5 pixels is made of thick oxide on p+ boron implant 

instead of the anti-blooming device. This part of the 
circuit uses then only two level polysilicon CCD gate 
structure. The area not used by the anti-blooming 
structure permits the CCD stage length to be reduced to 

10 13.2 microns versus 14 microns in the image zone. This 
achieves even a greater charge handling capacity than in 
the imaging pixel in order to have no saturation 
limitation in this area. This area is terminated by a 
specific gate controlling the charge transfer into the 

15 readout registers. 

In order to provide high speed charge 
transfer, interconnections are used to contact the pixel 
gates with aluminiim rails and aluminum buses run from 
the top of the array, driving both the image and the 

20 memory zones, on the anti-blooming structure in the 

image area, and on the thick oxide region in the memory 
zone . 

Because the entire array has a relatively 
large capacitance for each clock of the photosensitive 

25 array (Pi) of the whole imager 106 is in the range of 
which is difficult to drive through classical clock 
drivers such as the 0026 series. Accordingly, the array 
is divided into four sub-arrays of 256 columns, so that 
each can be readily driven at high speed by conventional 

30 clock drivers. This results in a typical frame transfer 
frequency of up to 2 Mhz, which is compatible with very 
fast imaging operation. 

The memory zone is provided with one CCD 
readout register for each of the four 256 x 1024 pixel 

35 sub-arrays. Each readout register is made of 256 stages 
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devoted to receive in parallel the signal charge coming 
from the associated pixel sub-array, plus 15 extra- 
stages needed for driving the charge towards the output 
amplifier, usable as electrical reference pixels as 
5 well- This structure is repeated four times along the 
last 1024 pixel line of the memory area. 

The four CCD readout registers use two 
polysilicon two phase structure, with buried channel, 
with the same 14 microns pitch as the imaging and fast 

10 framing zone 202. They are driven four in parallel by 
two clocks (L) , and terminates each 9X gate VGS, 
stopping on an output floating diode. A channel turn is 
used to allow the tapped structure to operate without 
disrupting the 14 microns pitch. The readout design 

15 structure is the classical "Floating Diode Readout" 
(FDRO) , which uses an output n+/p diode as a charge 
detector after reset through the clock (R) to the 
reference positive voltage VDR. Charge packets coming 
from the output register are dropped out one by one by 

20 the last clock (2L) into the floating diode well, over 
the output gate BGS, lowering the floating potential of 
that diode. This voltage is sensed by the respective 
output amplifier. Four output amplifiers are used for 
the circuit/ to deliver the video signal coming from the 

25 four output registers. They are designed for a 

bandwidth of 95 Mhz and low noise operation. Each of 
them is a double stage source follower NMOS amplifier. 

The output CCD registers are driven at 20.8 
Mhz, and the device may be read out in a progressive 

30 scanning mode with very high dynamic range (up to 80 dB 
w.r.t. noise level in darkness). The integration time 
may be set from zero to 15.9 ms., leading to the memory 
output rate of one memory block consisting of 16 
sequential subimages, per 16.6 ms. The imager zone 

35 frame transfer is operated at up to 1.3 Mhz, and the 
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applied voltages are set to the nominal value. 
Additional features of the imager, other than the high 
speed specialized masking and imaging process, are 
described in "THX 7887A: A New High- Frame -Rate 1024 x 
5 1024 Pixel CCD Sensor," G. Boucharlat et al., SPIE Vol. 
2273, pp. 255-263; see also, J. A. Cortiula, "THX 7897M: 
Capteur CCD 2048 x 2048 Pour Applications 
Astronomiques, " OPTO 93, Paris, May 11-13, 1993. 

The system of Figures 2-6 is based on the 

10 conversion of an existing frame transfer imager, . in 
order to minimize the design expense of creating new 
masks and other capital investment, because the existing 
design can be used with the addition of an additional 
metal or other opaque mask to produce the desired, 

15 restricted, active pixel pattern. Similarly, other 

existing imagers may be modified by applying a mask, to 
make the most cost-effective initial production of 
ultrafast imagers. For example, interline transfer 
imagers, full frame imagers, and frame transfer imagers 

20 may be utilized. The commercially available 2024 x 2024 
pixel Thomson CSF 7899 full frame imager is useful 
because of its large number of pixel cells, for making 
masked pixel designs of sparsely distributed active 
pixels. Such arrays were previously described as the 

25 arrays in which the active pixels are nominally 

separated or displaced in both column and row directions 
by 8 pixels, producing an imager which can store 64 or 
65 ultrafast sequential images. Use of gated 
photodiodes, which are used in many imagers such as 

30 those of the interline transfer type and which are not 
in the charge transport channel, permits the most 
effective electronic shuttering, to initiate imaging 
during the ultrafast sequence of imaging but to stop the 
imaging during the imager readout. Ultrafast CCD 

35 imagers may also be designed to take full advantage of 
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architectures in which the active photosites (preferably 
not in the charge transport /memory cell channels and are 
subject to electronic shuttering) represent a very small 
percentage of the area of the imaging array, and the 
5 analog memory cells associated therewith represent the 
major component of the imaging array. An example is the 
high speed system described in U.S. application no. 
08/423., 654, filed April 17, 1995, for High Speed CCD 
Imager, which, is incorporated herein by reference. 

10 • As illustrated in Figure 12, two or more 

ultrahigh speed cameras 1201, 1202, 1203 may be 
integrated into a stereo photdgrammetry system, through 
the use of a common, high precision trigger 1201. The 
cameras are adapted to receive a common triggering 

15 signal, which, together with a common frame gating and 
exposure program, provides synchronous stereo imaging 
from multiple cameras. The cameras 1201, 1202, 1203 may 
be fast framing solid state cameras with substantially 
film quality imagery, which are adapted to provide 

20 variable interframe periods from 100 microseconds to 100 
ns., with temporal gating capability down to 1 -10 ns. 
With pulsed laser illumination, the cameras may provide 
10-12 bit gray scale images with at least 4, and 
preferably at least 16 frames of on-chip image storage. 

25 Figure 13 is a schematic top view of a large 

format 2048 x 2048, and preferably 4096 x 4096 storage 
cell/pixel CCD imager 1206 of full frame architecture 
which is adapted for ultrahigh speed imaging for the 
camera system of Figure 12. The full frame CCD imager 

30 1206 has a large format of at least 2048 x 2048 storage 
cell/pixel size, and more preferably at least 4096 x 
4096 storage cell/pixel size, which features a four-port 
parallel output architecture for high data . throughput . 
It may be designed and fabricated substantially in 

35 accordance with the THX 7899M full field image sensor 
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which is commercially manufactured by Thomson CSF 
Semi conduct eurs Specif iques, Orsay, France. 

Electrical stimulus for the CCD imager is 
depicted in Figure 13 and is as follows: 

5 

• 1,2,3,4 P(n) The low speed four-phase clocks 
used for vertical image 

shift (clock rate = 1-10 Mhz) 

♦ ^ The antibloom gate clock 

10 V ^ The antibloom sink bias voltage 

*x,2L The high speed two-phase 

horizontal image shift clocks 

(clock rate = 20 Mhz) 
Vdd On-chip amplifier bias 

15 Vssa . On-chip amplifier ground return 

Vos(n) Video output 

The preferred imager itself comprises an approximately 

20 16 megapixel imaging zone formed by 4096 x 4096 square 
pixels (e.g., each pixel or storage cell measuring 14 
microns x 14 microns of which approximately 10 microns x 
14 microns is active photosite area) arranged along 
vertical CCD shift registers (in the direction of the 

25 arrows), allowing for simultaneous and controlled 
readout of the image zone analog data. 

In order to increase the output rate of the 
device, the imager has four output sections, each of 
which receives the analog charge packets from one-fourth 

30 (e.g., 1024) of the 4096 CCD vertacai transfer columns, 
as shown in Figure 12. During conventional operation, 
the imager would integrate the optical image input over 
each of the pixel cells in the image area for a 
predetermined period of time under control of the camera 

35 control system. The imager would be externally 
shuttered. Then the image would be shifted down 
vertically one row at a time until the entire image is 
directed through the respective output amplifiers. 

The special light shield modification 

40 comprises a metal mask of the type illustrated in Figure 
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3A or Figure 3B over the imaging section, which is 
opaque except for a precisely aligned 1024 x 1024 
pattern of pixel apertures. In conjunction with a 
customized CCD clocking sequence, this specialized light 
5 shield mask, permits ultrafast acquisition of 1024 x 
1024 subimages within the larger 4096 x 4096 array. By 
applying the specialized masking to the otherwise active 
pixels in the imaging zone, local image storage regions 
are created which eliminate the need to clock the entire 

10 image (4096 rows) out of the array before capturing the 
next image. In fact, appropriate design of the metal 
mask allows the 1024 x 1024 subimage to shift its data 
into a memory zone and prepare for the next image 
capture after only one vertical shift cycle. 

15 Since a vertical shift can be carried out in 

as short a time interval as 100 ns., this means that the 
1024 X 1024 subimager is capable of capturing new images 
at a rate of 10 MFPS until the locally created memory 
area has been exhausted. For a 1024 x 1024 subimage in 

20 a 4096 x 4096 image array, approximately 16 ultrahigh 
speed sequential subimages may be captured and stored 
for high precision, slow readout. 

Accordingly, the photosite and storage cell 
arrangement shown in Figure 3A is capable of capturing 

25 and storing 16 images of approximately IK x IK image 
size. Because the charge packets may be transported 
very rapidly along the transport channels (e.g., at four 
phase clocking speeds of up to 10 Mhz), the subimager 
actually captures a new image at up to a 10 million 

30 frame per second rate. The 16 successive images are 
interleaved as shown in Figure 3, with cells 16 being 
the first image, cells 15 being the second image, back 
to cells 1 being the most recent image. In conventional 
operation, the 4,096 x 4,096 imager would require 4,096 

35 vertical shifts in succession to read out an image 
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frame. 

In the high speed (10 million frames per 
second) mode of operation, this clocking pattern is 
stationary until initiated by the triggering signal. A 
5 conventional four phase clocking pattern is then 
initiated which provides a predetermined number of 
vertical parallel shifts which does not exceed the 
number of storage cells which are locally linearly 
associated with each photosite (here 16, including the 

10 photosite) . Within one cycle the clocking pattern is 
then abruptly stopped so that the subimages remain 
stored without overlap from image signals from adjacent 
photosites. The camera is then effectively shuttered, 
and remains in an idle state until readout is initiated. 

15 In some circumstances, it may be desirable to 

capture a burst of images at 10 million frames per 
second, pause for a length of time, capture a 1 million 
FPS burst, pause, capture at 500 KFPS, etc., as 
previously discussed. For example, the user might want 

20 to capture 6 frames at 10 million FPS, pause for 1 ms,, 
capture another burst of 6 at 1 million frames per 
second, pause for 10 microseconds and then fill the 
remaining memory buffer at 500 KFPS. This mode of 
operation may be provided as a programmable feature of 

25 the high speed solid state camera systems of the present 
invention. 

The transport electrodes of such a large image 
sensor are comparatively narrow and long, leading to 
relatively high resistive load in addition to the high 

30 capacitive load resulting from the large area of the 
imager* It is difficult to drive such a large image 
sensor at a high frame rate significantly in excess of 
one million parallel (vertical shift) cell transfers per 
second, without charge transfer inefficiency and other 

35 problems. To provide an imager structure for ultrafast 
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imaging at a frame rate in excess of five - ten million 
frames per second, the imager architecture may be 
modified by applying a custom light shield, utilizing a 
specialized clocking scheme, and by adding relatively 
5 large strapping or bus electrodes atop the light shield 
to facilitate high-speed operation. The strapping 
electrodes, in turn, are made possible because of the 
light shield and imager operation, because if a high 
proportion of the imager were used for photosites, the 

10 strapping electrodes would interfere with imaging at the 
photosites. 

As illustrated in Figure 14, the large 
strapping electrodes are provided atop and insulated 
from the metal mask applied to the imager. The wide 

15 buses are connected periodically to each of the 

respective same-phase transport drive electrodes by vias 
in accordance with conventional integrated circuit 
fabrication practice. Because they have a much smaller 
resistance, and a much smaller length-to-width ratio, 

20 very high speed operation is facilitated at higher 

transfer rates than those permitted by the four phase 
polysilicon transport electrodes alone, which have a 
width of only about 2,5 microns and are thin enough to 
permit imaging therethrough in the pixel cells. 

25 As illustrated in Figure 14, the space between 

the active pixel sites may advantageously be used for 
additional, relatively wide, highly conductive transport 
electrode busing, which can be used to apply the 
transport clock signals at periodic intervals (vias) to 

30 the underlying transport electrodes. Because these 
electrodes may be metallic and up tc ten or more times 
as wide and at least 10 times more conductive, as the 
polysilicon or silicide pixel transport electrodes (four 
to each pixel site in a four-phase transfer electrode 

35 structure such as used in the systems of Figures 1 and 



wo 97/46004 




PCT/US97/0T762 



-31- 



13), the resistive drive load may be greatly decreased, 
and the transport clock frequency may be increased to 5 
- 10 million frames per second, while retaining 
substantial pixel uniformity and charge transfer 
5 efficiency. 

To increase the light sensitivity to the final 
camera (since much of the active pixel area has been 
shielded from light) , a lenslet array is mounted over 
the masked CCD array to bring the fill factor up to at 

10 least 60%, and preferably at least about 85%. For 
example, in the pixel pattern of Figure 3A, a 
photoactive aperture exists only once every fourth 
pixel. Assuming a basic pixel pitch of 14 microns, the 
fill factor would be about 1/16 or 6%. By applying 

15 lenslets over and focussed on each of the sensitive 
apertures, which are about 55.5 microns in diameter 
(with a 1 micron spacing between lenslets for 
manufacturing purposes), the effective fill factor is 
brought up to over 90%. 

20 A significant advantage of this solid-state 

architecture is that after acquiring 16 images at up to 
10 million frames per second, the "store" of captured 
frames may be read out at relatively low data rates 
allowing for the very high dynamic range. Because 

25 readout data rates can be kept to relatively low rates 
such as 20 megapixels/second, standard (commercially 
available) high performance readout circuitry including 
analog-to-digital converters and frame grabbers can be 
used for data acquisition, rather than expensive 

30 ultrahigh speed data capture systems. 

High speed temporal gating which is 
independent of the frame rate may be accomplished by 
pulsed laser illumination and/or by high speed gated 
intensifiers. Copper vapor lasers exist with pulse 

35 durations on the order of 1 ns. gating. While this 
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"gate" duration is sufficient for most high speed 
imaging, the repetition rate of the pulsed laser source 
is typically limited to 10 's of kilohertz. Gated 
intensifiers offer an alternative path which provides 
5 not only temporal gating, but image intensification (a 
significant advantage in high speed imaging 
applications) . 

Intensifiers with high resolution of up to 641 
p/mm. can be achieved on an 18 mm. intensifier cell 

10 which provide repetition rates in excess of 10 Mhz and 
gating resolutions on the order of 1-5 ns. 

The cameras of Figures 1 and 12 are designed 
to operate in wideband panchromatic mode, or in 
monochromatic mode with an external color filter or 

15 light source. Illustrated in Figure 15 is the masking 
for a color version of an ultrahigh speed imager. In the 
imager 1500 of Figure 15, a mosaic of microscopic red 
color filter dots 1501, microscopic green color filter 
dots 1502 and microscopic blue color filter dots 1503 

20 are fabricated on top of the imager array before 
applying the light shielding mask as shown in the 
Figure. As shown in Figure 15, 3 columns are utilized 
to store image data for each RGB pixel array. The 
pixels may be rectangular (e.g., with a length to width 

25 ratio of up to 3:1), to make the overall RGB pixel area 
more square, A large microlens over each pixel is used 
to focus the light from a circular area onto the RGB 
pixel pattern. 

The rate of image data captured by very high 

30 speed imaging systems such as the two dimensional array 
illustrated in Figure 2 limits the number of frames and 
image resolution which can be accomplished by such two 
dimensional arrays. 

In order to provide for additional image 

35 memory locally associated with each active photosite, 
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the present invention is also directed to CCD imager 
systems which utilize three dimensional stacked layers 
of parallel edge-input CCD imaging arrays such as an 
edge-sensitive CCD 702 shown in Figure 7 to form an 
5 integral imager, memory, and output or processing system 
such as a system 802 shown in Figure 8. Such stacked 
arrays can store a higher number of image frames, and 
may also process the sequential images acquired at such 
high speeds to compress or selectively simplify the data 

10 output requirements of the syistem: Imager and camera 
systems based on extraordinarily compact imager/memory 
stacks formed by such chips can provide significantly 
enhanced performance. The imager/memory stack may be 
assembled by stacking individual CCD memory chips to 

15. form an imaging face and an output face (Figure 8). 

Each individual memory chip may be driven at 1 - 10 Mhz, 
which is relatively slow for CCD memory operation, but 
which provides ultrafast imaging operation because of 
the fully parallel processing design. The chips have a 

20 linear array of edge-sensitive input pixels, each of 
which is clocked to its own CCD memory channel, which 
may be 1,000 or 2,000 memory cells or more long. The 
memory channels are multiplexed and clocked to one or 
more conventional high speed output amplifiers, for 

25 clocking the image data off-chip, as shown in Figure 7. 

When the edge- sensitive CCD memory chips of 
Figure 7 are stacked vertically, the linear photosites 
at the edges of each chip form a 2-dimensional pixel 
array at the imager face. In addition, the associated 

30 CCD memory channels form a compact, very high-density 
3-D memory which operates fully in parallel to receive 
the data which is clocked out from each corresponding 
edge-photosite at 1-10 Mhz. As indicated, the imaging 
face is formed by the edge-sensitive linear pixel array 

35 on each chip, which may include optical waveguides and 
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lenslets to improve photoresponse. The x-spacing (see 
Figure 8) of the pixels is fabricated directly on each 
chip. The y-spacing of the pixels depends on the 
thickness of the chip wafers ^ which can range from 10 
5 microns for ultrathin wafers to 300 microns or more for 
"standard" wafers. A lenslet array with off -axis 
lenslets can be used to effectively decrease the y-axis 
pixel spacing {while increasing the x-axis spacing) . 

Alternatively, a separate imager such as a 

10 backside thinned, or silicon-on-sapphire imager, can be 
directly indium bumped to edge-sensitive CCD input zones 
of the stacked 3-D CCD memory cube. CID image sensors 
can be readily biamped to respective charge mode 
detectors on the stacked memory chips, and accordingly 

15 are desirable sensors for such 3-D bumped imager arrays. 
Because the photosites are clocked directly in parallel 
to the corresponding CCD memory channels, the 
bump-mounted imager can be operated at very high frame 
rates corresponding to the 1-10 Mhz rate of the memory 

20 CCD channels. 

Bump-mounting of a silicon-on-sapphire imager 
to the 3-D CCD memory stack permits closer y-pixel 
spacing when "thick" wafers are stacked. For example, 
by multiplexing 8 pixels of a bump-mounted imager to 

25 each CCD memory channel of a "stack" of 200 micron thick 
CCD memory chips, a 25 micron pixel spacing is produced. 
For IK-long CCD memory channels, this provides memory 
storage for 125 full frames of the image for 2K long CCD 
memory channels, 250 full frame images can be stored in 

30 the 3-D array, which for a 512 x 512 imaging array 

multiplexed onto 64 stacked chips, constitutes a memory 
capacity of over 65 megabytes. 

As shown in Figure 9, bump-mounting may also 
be used for interconnects at the output face of the 

35 imager/memory cube. This permits separate control of 
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high-speed imager drivers, high-speed CCD memory channel 
drivers, and slower speed output drivers. The compact/ 
stacked geometry is suitable for operation in triggered 
burst mode, in which a triggering event initiates 
5 high-speed image capture and storage of 1,000 or more 
image frames in the analog CCD memory along the "z" 
axis. The 1,000 or more frames of image data may then 
be clocked or "read" out for processing at a second, 
slower data rate. 

10 The stacked z-axis CCD memory chip geometry of • 

this disclosure has advantages of full design efficiency 
for true parallel operation, separate imager, memory and 
output driver connection capability, enormous functional 
compactness and 3-D density of operational circuitry, 

15 small size, light weight, compatibility with image 
intensifiers and fill-factor enhancing lenslet arrays 
selectability of "good" chips for further assembly in 
defect-free larger array, and high yield fabrication of 
simple CCD memory chips with one linear imager array. 

20 X-axis pixel separations are fabricated 

on-chip. However y-axis tolerances are subject to wafer 
thickness and stacking uniformity, while z-axis toler- 
ances are subject to wafer die-cutting and alignment 
tolerances. Precise deep-etch techniques such as the 

25 "black silicon" vertical etching technique are used as 
opposed to wafer-saw techniques to separate the CCD dice 
for stacking, and to avoid silicon lattice damage which 
would result in excessive dark current ["The Black 
Silicon Method", H. Jansen et al., J. Micromech. 

30 Microeng. 5, pp. 115-120 (1995)]. Various alignment 

methods, including "pillar-recess" stacking as shown in 
Figure 10 and optical flat assembly may be used for 
z-axis alignment. 

Bonding of even deeply grooved silicon wafers 

35 is easily accomplished using photoresist or anode 
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bonding of silicon wafers with a thin layer of photo- 
resist or R.F. sputtered glass at temperatures of less 
than 450* C. A processing potential of about one volt 
is sufficient to anode-bond silicon wafers with glass 
5 layers of less than 100 nm, [V. Spiering, et al., 

"Sacrificial Wafer Bonding for Planarization After Very 
Deep Etching, " Journal of Microelectromechanical 
Systems, Vol- 4, pp. 151-157 (1995)]. 

Appropriately aligned mirror-polished silicon 

10 surfaces may also be bonded (in a. monocrystalline 

manner) by wafer bonding techniques in which surface 
silanol groups dehydrate and remaining oxygen atoms 
dissipate at elevated temperatures. Low temperature 
bonding methods have recently been developed for Si/Si 

15 bonding at temperatures of less than 400** C,, which may 
be suitable [Q. Tong, "Low Temperature Wafer Direct 
Bonding, " Journal of Microelectromechanical Systems, 
Vol. 3, pp. 29-35 (1994)], and may be attempted for 
bonding of frontside mounting posts to the wafer 

20 bacJcside, such as shown in Figure 7. 

The heat dissipation of a single CCD memory 
chip operating at 1 Mhz clocJcing rate using 
low-resistance aluminum cloclc electrodes is typically 
about 2 watts, and a 3-D "stacJc" of 512 such chips will 

25 accordingly generate heat at about 1000 watts in a 2 cm. 
cube. This heat output is controlled by (a) designing 
the drive circuitry to only operate while the imager is 
functioning (e.g., the total heat for 1,000 frames at 1 
megaframes/second is less than 0.001 x IK watts = 10 

30 watts), and (b) removing the heat using a 3-D internal 
coolant network. The high-volume density of circuitry 
generates significant heat, which may be removed by 
passive integral heat pipe or forced coolant flow. As 
shown in Figure 10, a pillar positioning chip-stack may 

35 be cooled by a coolant-flow system through the y-z 



WO 97A 



460(1^^ 



wo 97/46004 




PCT/US97/07762 



-37- 



plane. Similarly, the CCD memory channels, being 
narrower than the imager pixels, can accommodate 
through-chip coolant holes etched on-chip by the 
deep-etching parallel-wall techniques used for micro- 
5 mechanical device fabrication and chip separation. 

Microheatpipe arrays may be used as efficient 
backside heat spreaders or removers, by reducing the 
thermal path between the frontside heat sources and a 
thermal sink [A. Mallik, "Fabrication of Vapor-Deposited 
10 Micro Heat Pipe Arrays as an Integral Part of Semi- 
conductor Devices," Journal of Microelectromechanical 
Systems, Vol. 4, pp. 119-131 (1995)]. Sealed 
microheatpipe arrays may be fabricated on the back side 
of thicker chip wafers as wickless, noncircular channels 
15 with a cross sectional hydraulic diameter of 10 - 50 

microns, and a length of up to 2 centimeters. As little 
as two percent surface coverage produces a 4 0% increase 
in heat removal. Even greater heat removal can be 
accomplished by forced liquid circulation through the 
20 chip stack. 

The compact 3-D stacked design may be used 
with conventional image intensifiers, and by using 
lenslets, a fill factor of BOS or more can be provided. 
The design is also fully compatible with image intensi- 
25 fiers which produce a direct electron output, by using 
electron-absorbing edge-sensitive input "pixels". 
Application of this electron bombarded sense mode can 
provide an effective optical gain of about 5000:1 over 
conventional CCD imagers. 
30 It will be appreciated that although various 

aspects of the invention have been described with 
respect to specific embodiments, alternatives and 
modifications will be apparent from the present 
disclosure, which are within the spirit and scope of the 
35 present invention as set forth in the following claims. 
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WHAT IS CLAIMED IS: 

1. A solid state imaging array comprising an 
array of photodetectors having locally associated there- 
with a storage plurality of sequentially operable analog 
memory cells^ capable of high speed successive imaging. 

2. A solid state imaging array in accordance 
with claim 1 in which the locally associated memory 
cells are linearly associated with each photosite along 
a CCD charge transport channel. 

3- A solid state imaging array in accordance 
with claim 1 in which the photodetectors are arranged in 
a two dimensional imaging zone, and at least some of the 
analog memory cells associated therewith are oriented 
5 with a directional component orthogonal to the two 
dimensional imaging zone to form a three dimensional 
imaging and storage assembly. 

4. A solid state imaging array comprising an 
array of photodetectors having associated therewith 
local memory wherein said array may be scanned at a 
first rate and at a second rate to provide an effective 
first frame rate and an effective second frame rate with 
exposure duration control. 
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